Can the bias in algorithms help us see our own?

🌈 Abstract

The article discusses how algorithms can reveal and amplify human biases, but also how they can be used as tools to help people recognize and correct their own biases.

🙋 Q&A

[01] Can Algorithms Help Us See Our Own Biases?

1. What did the research by Carey Morewedge and his collaborators find?

• The research found that people are more likely to recognize biases in decisions made by algorithms than in their own decisions, even when the decisions are the same.
• Participants saw as much bias in the decisions of algorithms as they did in the decisions of other people.
• Participants were also more likely to correct for bias in the decisions they thought came from algorithms, compared to their own decisions.

2. How do the researchers explain this phenomenon?

• The researchers attribute this to the different evidence people use to assess bias in others versus bias in themselves.
• When analyzing others' decisions, people focus on the outcome, which makes it easier to perceive bias.
• When assessing their own decisions, people have insight into their thought process and are more likely to find other factors that justify their decisions, making them less likely to recognize their own biases.

3. What are the implications of this research?

• The research suggests that algorithms can be used as "mirrors" to help people better see and correct their own biases.
• While algorithms can codify and amplify human biases, they can also be a tool to make people more aware of their own biases and motivate them to correct them.

[02] How Algorithms Can Reveal Structural Biases

1. What example does the article provide of how algorithms can reveal structural biases?

• The article discusses how Amazon's hiring algorithm boosted résumés it perceived to come from male applicants and downgraded those from female applicants. This revealed a gender bias in Amazon's existing hiring decisions, even though only 39% of Amazon's workforce were women at the time.

2. How do the researchers explain this?

• The researchers argue that if the algorithm had a gender bias, "it's because Amazon's managers were biased in their hiring decisions."
• Algorithms can "codify and amplify human bias," but they can also "reveal structural biases in our society" that may not be easily observable at an individual level.

3. What is the significance of algorithms revealing structural biases?

• The article suggests that when algorithms aggregate many individual decisions, they can expose biases that are difficult to see in isolated decisions.
• This can help identify systemic biases within organizations and society, which is an important step towards addressing them.